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Abstract 

In evolutionary games the fitness of individuals is not constant but depends on the rela- 
tive abundance of the various strategies in the population. Here we study general games 
among n strategies in populations of large but finite size. We explore stochastic evolution- 
ary dynamics under weak selection, but for any mutation rate. We analyze the frequency 
dependent Moran process in well-mixed populations, but almost identical results are found 
for the Wright-Fisher and Pairwise Comparison processes. Surprisingly simple conditions 
specify whether a strategy is more abundant on average than 1/n, or than another strategy, 
in the mutation-selection equilibrium. We find one condition that holds for low mutation 
rate and another condition that holds for high mutation rate. A linear combination of these 
two conditions holds for any mutation rate. Our results allow a complete characterization 
of n X n games in the limit of weak selection. 
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1 Introduction 



Evolutionary game theory is the study of frequency dependent selection (May- 
nard Smith and Price, 1973; Maynard Smith, 1982; Hofbauer and Sigmund, 1998, 
2003; Nowak and Sigmund, 2004). The individuals of a population can adopt one 
of several strategies, which can be seen as genotypes or phenotypes. The payoff for 
each strategy is a linear function of the relative frequencies of all strategies. The 
coefficients of this linear function are the entries of the payoff matrix. Payoff is 
interpreted as fitness: individuals reproduce at rates that are proportional to their 
payoff. Reproduction can be genetic or cultural. 
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Evolutionary game theory provides a theoretical foundation for understanding hu- 
man and animal behavior (Schelling, 1980; Maynard Smith, 1982; Fudenberg and 
Tirole, 1991; Binmore, 1994; Aumann andMaschler, 1995; Samuelson, 1997). Ap- 
plications of evolutionary game theory include games among viruses (Turner and 
Chao, 1999, 2003) and bacteria (Kerr et al., 2002) as well as host-parasite inter- 
actions (Nowak and May, 1994). Cellular interactions within the human body can 
also be evolutionary games. As an example we mention the combat between the im- 
mune system and virus infected cells (Nowak et al., 1991; May and Nowak, 1995; 
Bonhoeffer and Nowak, 1995). The ubiquity of evolutionary game dynamics is not 
surprising, because evolutionary game theory provides a fairly general approach to 
evolutionary dynamics (Nowak, 2006). There is also an equivalence between fun- 
damental equations of ecology (May, 1973) and those of evolutionary game theory 
(Hofbauer and Sigmund, 1998). 

Let us consider a game with n strategies. The payoff values are given by the n x n 
payoff matrix A = [aij]. This means that an individual using strategy i receives 
payoff ttij when interacting with an individual that uses strategy j. For understand- 
ing a game it is useful to explore whether any of the strategies are Nash equilibria 
(Nash, 1950; Maynard Smith, 1982; Taylor and Jonker, 1978; Cressman, 1992). 
Strategy A; is a strict Nash equilibrium if a^fc > a^fc for alH 7^ k. Strategy /c is a 
Nash equilibrium if akk > o-ik for all i. Another useful concept is that of an evolu- 
tionarily stable strategy (ESS) (Maynard Smith and Price, 1973; Maynard Smith, 
1982, 1974). Strategy k is ESS if either (i) akk > ciik or (ii) akk = ciik and aki > an 
holds for all i ^ k. We have the following implications: if /c is a strict Nash equilib- 
rium then it is an ESS; if k is an ESS then it is a Nash equilibrium. Both Nash and 
ESS, however, give conditions on whether a strategy, which is played by the ma- 
jority of players, outperforms all other strategies. Hence they identify the 'favored' 
strategy based on its performance at large frequencies. 

The traditional approach to evolutionary game dynamics uses well-mixed popu- 
lations of infinite size. In this case the deterministic selection dynamics can be 
described by the replicator equation, which is an ordinary differential equation de- 
fined on the simplex Sn (Taylor and Jonker, 1978; WeibuU, 1995). Many interest- 
ing properties of this equation are described in the book by Hofbauer and Sigmund 
(1998). 

More recently there have been efforts to study evolutionary game dynamics in 
populations of finite size (Riley, 1979; Schaffer, 1988; Kandori et al., 1993; Kan- 
dori and Rob, 1995; Fogel et al., 1998; Ficici and Pollack, 2000; Schreiber, 2001; 
Nowak et al., 2004; Taylor et al., 2004; Wild and Taylor, 2004; Traulsen et al., 
2005). For finite populations a stochastic description is necessary. Of particular in- 
terest is the fixation probability of a strategy (Nowak et al., 2004; Antal and Scheur- 
ing, 2006; Lessard and Ladret, 2007): the probability that a single mutant strategy 
overtakes a homogeneous population which uses another strategy. When only two 
strategies are involved, the strategy with higher fixation probability is considered 
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to be more 'favored' by selection. We can take a game of n strategies and analyze 
all pairwise fixation probabilities to find which strategies are favored by selection 
(Imhof and Nowak, 2006). This concept, in some way, compares strategies at all 
relative frequencies during the fixation process, as opposed to the Nash and ESS 
conditions. 

The study of fixation probabilities, however, is only conclusive for small mutation 
rates, which means most of the time all players use the same strategy. In this paper, 
we propose a more general way of identifying the strategy most favored by selec- 
tion: it is the strategy with the highest average frequency in the long time average. 
For brevity we call throughout this paper the average frequency of a strategy in 
the stationary state its abundance. The criteria for higher abundance can be used 
for arbitrary mutation rates. Moreover, for small mutation rates this criteria can be 
formulated in terms of pairwise fixation probabilities. 

In particular, we focus on stochastic evolutionary dynamics in populations of finite 

size A^, although for simplicity we shall consider the large (but still finite) popula- 
tion size limit. Evolutionary updating occurs according to the frequency dependent 
Moran process (Nowak et al., 2004; Taylor et al., 2004), but the Wright Fisher pro- 
cess (Imhof and Nowak, 2006) and the Pairwise Comparison process (Szabo and 
Toke, 1998; Traulsen et al., 2007) are also discussed; the details of these processes 
are explained in the next sections. In addition, we assume that individuals repro- 
duce proportional to their payoffs but subject to mutation with probability u > 0. 
With probability 1 — u the imitator (or offspring) adopts the strategy of the teacher 
(or parent); with probability u one of the n strategies is chosen at random. 

We study the case of weak selection. For the frequency dependent Moran process, 
the payoff of strategy i is given by = 1 + ^tTj, which is the baseline payoff, 
1, plus the payoff tt^ of strategy i obtained in the games, weighted by the inten- 
sity of selection 5 > 0. Weak selection means 5 -C 1/iV. In this case, although 
the frequencies of the strategies can widely fluctuate in time, all strategies have 
approximately the same abundance (average frequency), 1/n, in the stationary dis- 
tribution of the mutation- selection process. We are interested in the deviation from 
this uniform distribution. To calculate this deviation we use a perturbation theory 
in the selection strength, 5. Here we follow the methods developed in Antal et al. 
(2008) for studying two strategies in a phenotype space. Perturbation studies can 
also be found in Rousset (2004) for subdivided populations. 

In this paper we study n-strategy games in a well mixed population of players. 
We consider that selection favors a strategy if its abundance (average frequency) 
exceeds 1/n. Conversely, selection opposes a strategy, if its abundance is less than 
1/n. We establish the following results. For low mutation probability (u <^ 1/N), 
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we find that selection favors strategy k if 




n 



X](afeifc + Ofei — Qife — Oil) > 0. 



(1) 



For high mutation probability {u 1/-/V), selection favors strategy k if 



n n 



(2) 



1=1 j=i 



For arbitrary mutation probability the general expression for selection to favor strat- 
egy k is 



All these results hold for large but finite population size, 1 <^ N <^ 1/S. They 
allow a complete characterization of n x n games in the limit of weak selection. 
The equilibrium frequencies of each strategy are also given in the paper. 

We can gain some qualitative understanding of our low (1) and high (2) mutation 
rate results. For low mutation rates, most of the time, all players use the same 
strategy until another strategy takes over. There are only two strategies involved 
in a takeover. A single A; -player fixates in all i-players with a higher probability 
than a single i-player into /c-players, if a^fc + Ofei — cbik — an > (Nowak, 2006). 
For only two strategies present, a higher fixation probability for k means that it is 
more abundant. Hence strategy k is the most abundant among all strategies if it 
fixates well against all strategies, which then explains expression (1). Conversely, 
for high mutation rates the frequencies of all strategies are close to 1/n all the 
time. Hence the payoff of strategy k is roughly — 1 + (S/n) Y.j=i o-tj- One has 
to compare this payoff to the average payoff of the population (1/n) Y.i fu which 
leads to expression (2). 

The rest of the paper is structured as follows. In Section 2, we derive the general 
conditions for strategy abundance for any mutation rates. Section 3 provides three 
concrete examples. Possible extensions of our method to strong selection, more 
general mutation rates, the Wright-Fisher and the Pairwise Comparison processes 
are discussed in Section 4. We summarize our results in Section 5. 



2 Perturbation method 

Let us consider a well mixed population of players. Each of them plays one of the 
n>2 strategies. The state of the system is described by the n-dimensional column 



Lfe + NuHk > 0. 
Strategy k is more abundant than strategy j if 



(3) 



Lk + NuHk > Lj + NuHj . 



(4) 
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vector X, where is the number of players using strategy i. The frequencies of 
strategies are x = X./N. The payoff matrix is given by the n x n matrix A = [aij], 
where a^j is the payoff of an i -player playing against a j -player. The payoff of an 
individual using strategy i is fi, and the column vector f is given by f = 1 + SAx. 
Here 5 > is the selection strength, and li — 1 (for all i) is the baseline payoff. 
The term AX/A^ = Ax in the player's payoff stands for the average contribution 
from all other players through the game. We included self interaction here, since 
it does not make a difference in the large N limit. The total payoff of the whole 
population is F = X^f = A'^(l + 5x^Ax). We assume weak selection throughout 
this paper, by which we mean that SN <^ 1. The need for such weak selection (as 
opposed to 5 ^ 1) shall become clear at the end of this section. 

The dynamics of the system is given by the frequency dependent Moran process. In 
each time step a randomly chosen individual is replaced by a copy of an individual 
chosen with probability proportional to its payoff. The offspring inherits the par- 
ent's strategy with probability 1 — or adopts a random strategy with probability 
u>0. 

We shall show below that the condition for strategy k to be more abundant than 

the average 1 /n is equivalent to having a positive average change of its frequency 
during a single update step. Hence we start deriving this latter quantity. In state X, 
the average number of offspring (fitness) of a A;-player due to selection is tUk = 
1 — 1/N + fk/F. We also included the parent among the offspring, which explains 
the leading 1 on the right hand side. The term —1/N describes its random death, 
while the term fk/F stands for the proliferation proportional to payoff. For 5 — > 0, 
the fitness can be written as 

ujk^l + 5N-^[{Ax)k - x^Ax] + 0{S^N-^), (5) 

In one update step, the frequency of A; -players changes on average due to selection 
by 

Axf = Xkouk -xu = 5A4'^[1 + 0{5)i (6) 
where the first derivative with respect to 5 is 

Axi') = N-^Xk[{Ax)k - x^Ax]. (7) 

The state of the system, X, changes over time due to selection and mutation. In the 
stationary state of the Moran process we find the system in state X with probability 
Ps^X). This stationary probability distribution is the eigenvector with the largest 
eigenvalue of the stochastic transition matrix of the system (van Kampen, 1997). 
The elements of the transition matrix depend on 8 only through f = 1 + 0{8). 
Note that there is no N dependence in the correction term, since both A and x 
are independent of A^. Consequently, the stationary probabilities are continuous at 
5 — and we can write them as -P5(X) = P5=o(X)[l + 0{5)] for any state X. 
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Hence by averaging Axf^ in the stationary state, in the leading order in S we obtain 

{Axf)s = ^ Axf A(X) = 5^ Aa;«n=o(X) x [1 + 0{6)]. (8) 

X X 

Thus, we can describe the stationary state of the system for small S by using the 
stationary distribution in the absence of selection, S — 0. Since the correction term 
is independent of N, the above formula remains valid even in the large population 
size limit. Using expression (7) for Ax^j^\ the average change due to selection in 
the leading order can be written as 



(9) 



{Axf)s = 57V-i(x,[(Ax)fe - x^Ax]) 

j hi 
where (•) denotes the average in the neutral stationary state (S — 0). 

So far we have only considered selection. By taking into account mutation as well, 
the expected total change of frequency in state X during one update step can be 
written as 

Axf = Axf (1 - + ^ (^1 - X,) . (10) 

The first term on the right hand side describes the change in the absence of muta- 
tion, which happens with probability 1 — u. The second term stands for the change 
due to mutation, which happens with probability u. In this latter case the frequency 
Xk increases by 1/nN due to the introduction of a random type, and decreases 
by Xk/N due to random death. In the stationary state the average total change of 
the frequency is zero, {Ax]^^)s = 0, that is selection and mutation are in balance. 
Hence by averaging (10) we obtain the abundance (average frequency) in the sta- 
tionary state expressed by the average change due to selection as 

1 1—7/ 

{x,)s^- + N——{Axf)s. (11) 

We emphasize that this relationship is valid at any intensity of selection, although 
we are going to use it only in the weak selection limit. From (1 1) it follows that the 
condition {xk)5 > 1/n is in fact equivalent to 

{Axf)s > . (12) 

That is, for strategy k to be more abundant than the average, the change due to 
selection must be positive in the stationary state. Hence, as we claimed, instead of 
computing the mean frequency, we can now concentrate on the average change (9) 
during a single update step. 

To evaluate (9) we need to calculate averages of the form (x^Xj) and (x^XiXj). 
Since in the neutral stationary state all players are equivalent, exchanging indexes 
does not affect the averages. For example {xiXi) = {x^x^}, and {X1X2X2) — 
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(xiXsx^). By taking into account these symmetries, only six different averages ap- 
pear in (9) 



(xixi) = 
{X1X2) = 

(xiXiXi) = 

{XIX2X2) = 



XjXi 



rf rf rf \ 

X. Y' ■ Y' ■ \ 

XjXjX^ 



(13) 



for all A; 7^ i 7^ j 7^ A;. Equation (9) then takes the form 

Nb'^^l^x'f)?, = {xixi)akk + {X1X2) ttki - {xixixi)akk 

i,iy^k 

- {X1X2X2) i^ki + an + ttik) - (xiXaXg) ^ 



(14) 



i,i^k 



kj^ijtjj^k 



Note that {xiX2X-i) is not defined for n = 2, but in that case the last sum in (14) is 
zero anyway. Hence the following derivation is valid even forn = 2. By removing 
the restrictions from the summations in (14), we can rearrange this expression into 

N5~^{Ax^^^)s = akk[{xixi) - {X1X2) - (xiXiXi) + 3(a:iX2X2) - 2(a:iX2X3)) 

+ {X1X2) Y (^ki + {{X1X2X3) - {X1X2X2)) Y(^ki + O-ii + O-ik) 
i i 

- {X1X2X3) Y 

(15) 



Let us now interpret these average quantities. We draw j players at random from 

the population in the neutral stationary state, and define Sj as the probability that 
all of them have the same strategy. We have {xi) = nr^ because under neutrality 
a player has 1/n chance of having strategy one out of n possibilities. Moreover, 
we have (xiXi) = S2n~^, because the first player has strategy one with probability 
1/n and the second player uses the same strategy with probability S2- Similarly 
(xiXiXi) — ssn~^ holds. The remaining averages that appear in (15) can be written 
as 

{X1X2) = ((1 - J2 = (xi) - (xiXi) - (n - 2){xiX2) 

2<i<n 

{X1X2X2) = ((1 - Yl ^1)^2X2) = {xiXi) - (xiXiXi) - (n - 2)(xia;2X2) 

2<i<n 

{X1X2X3) = ((1 - Y Xi)x2X3) = {X1X2) - 2{xiX2X2) - {u - 3) (a;ia;2a;3) 

2<i<n 

where we used the normalization condition J2iXi — 1, and the symmetry relations 
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(13). Thus, we can express all the averages in (13) in terms of only two probabili- 
ties, S2 and S3 



{XIX2) 

(XiXiXi) 

{XIX2X2) 

{xiX2Xs) 



n 

S2 

n 

I-S2 
nin — 1) 

S3 ' (16) 

n 

S2 - 

n{n — 1) 

1 - 3,S2 + 2,S3 

n{n-l){n-2) ' 



We note again that forn = 2 the last expression is ill defined, but it is not needed 
in that case. 

Up to this point everything was calculated for finite N. Although further discussion 
for finite N is possible, it becomes quite unwieldy; hence for simplicity we consider 
only the large N limit from here on. In Appendix A we calculate the values of S2 
and S3 for A?^ :§> 1, which are given by (A.3) and (A.7), respectively. By substituting 
these expressions into (16) we arrive at 

(xiXi) = n{2 + fj,){n + ii)C 

(X1X2) = /i(2 + fj,)nC 
{xiXiXi) = {n + ji){2n + ii)C (17) 
{X1X2X2) = ^i{n + /i)C 

{xiX2X^) = l/C , 

where C = [Nn^(l + /x) (2 + /x)] ~^ and = Nu is the rescaled mutation rate. With 
these correlations, (15) takes the form 



— ^1 — = l^n'^akk + /^(2 + ii)n ^ aki - i^n Y^{aki + an + aik) - % ' 



where rearranging the terms leads to 



{Ax 



sel\ 

k )s 



cs 
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By defining 



Lk — — y^i^kk + O'ki — (^ik — (^ii) 

n ^ 

1 (18) 

Hk = — 'Y^{o^ki — O'ij), 



we finally arrive at our main result 

n7V(l + /.)(2 + /.) • ^'^^ 

This expression is valid in the limit of large population size ^ 1, for weak 
selection N5 <^ 1, with /i = Nu being constant. Condition (12) for strategy k to 
be more abundant than the average 1/n is simply + jjiHk > as we already 
announced in (3). In the low mutation limit {ji 0) the condition for abundance 
becomes Lk > 0, while in the high mutation limit (// — > oo) it is Hk > 0. As a 
consequence of (1 1), strategy k is more abundant than strategy j if Lk+^iHk > Lj+ 
jiHj. Note that any finite mutation probability u corresponds to the high mutation 
rate limit — > oo for our iV — > oo limit. 

By substituting (19) into (11) we obtain the abundances (average frequencies) in 
the weak selection stationary state 



{Xk)5 = - 

n 



1 + dN[l — uj- 



{l + Nu){2 + Nu) 



(20) 



This expression becomes exact in the N — > oo, A'^^ — > limit, if Nu = is kept 
constant. It becomes clear at this point, that although we only used 5 <^lX.o derive 
(19), we actually need SN <^ 1 to have frequencies close to 1/n in (20). 



2.1 Special case: Two strategies 



For only two strategies (n = 2) the general formula (19) leads to 

{Axf)s = ^ ^^-) + «i2 - «2i - 022). (21) 

The peculiarity of the two strategy case is that the condition for higher abundance 
(mean frequency) (12) of strategy one 

^11 + 0,12 — 021 — 022 > (22) 

does not depend on the mutation probability u. It has been shown in (Antal et al., 
2009) that very similar conditions hold for finite population size. With self interac- 
tion we obtain the same result, but when self interaction is excluded, the condition 
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becomes 

(an + ai2 - 021 - ^22)^ - 2aii + 2a22 > (23) 
This condition does not depend on the mutation probability u either. Moreover, the 
above conditions are also valid for arbitrary strength of selection for a general class 
of models, in particular for the Moran model with exponential payoff functions 
or for the Pairwise Comparison process (Antal et al., 2009). Note that this law is 
well known for several models in the low mutation rate limit (Kandori et al., 1993; 
Nowak et al, 2004). 



2.2 Low mutation rates 



There is an intimate relationship between our conditions for high abundance and 
fixation probabilities for low mutation rates fi <^ 1. In this limit, most of the time 
all players follow the same strategy, and rarely a single mutant takes over the entire 
homogeneous population (fixates). During fixation only two types of players are 
present. The fixation probability pij is the probability that a single i-player over- 
takes a population of j-players. Hence we have effectively n states of pure strate- 
gies, where a state of pure strategy j changes to a state of pure strategy i at rate 

Let us first consider n — 2 strategy games, where we label the two strategies as k 
and i. In the stationary state there are rare transitions between pure A; -player and 
pure i-player states, and {xk)pik — {xiiPu with {xk) + {xi) — 1. Hence we can 
write 



{Xk) 



N , 
1 + y(Pfci 



Pik) 



(24) 



since all fixation probabilities are 1/N in the leading order of 5. On the other hand, 
the abundance (20) for two strategies and low mutations becomes 



{Xk) 

Consequently, we can express SL^ as 



1 



N 



5U 



(25) 



^ik ^ii) — Pki Pik- 



(26) 



This equality can also be derived independently from the exact expression of the 
fixation probability (Nowak et al., 2004) 



Pki — 



N 



SN 

1 + —^{O'kk + '^O'ki — O-ik — '^O'ii) 



(27) 



For n strategies, by using (1) and (26), we can express with pairwise fixation 
probabilities as — {2/5n) J^i Pki — Pik- The condition > for strategy k to 
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be more abundant than 1/n can be written as 

Y^Pki>Y^ Pik (28) 

i i 

This condition can be interpreted as follows: strategy k is more abundant than 1/n 
in the low mutation rate limit if the average fixation probability of a single A;-player 
into other pure strategy states is larger than the average fixation probability of other 
strategies into a pure strategy k population. For these averages we take all strategies 
with the same weights. 



3 Examples 



Here we provide three applications of our results for three strategy games. First in 
3.1 we study the effect of Loners on Cooperators and Defectors. Then in 3.2 we 
show how mutation alone can make a strategy more abundant. Finally in 3.3 we 
study the repeated Prisoner's Dilemma game. 



3.1 Cooperators, Defectors, Loners 




To see the difference between our weak selection and a traditional game-theoretic 
approach, let us consider the following example. We start with a Prisoner Dilemma 
game between cooperators (C) and defectors (P), given by the payoff matrix 



(29) 



Clearly, defectors dominate cooperators, so we expect that defectors are more abun- 
dant in a stationary state. Indeed, from condition (22) we obtain 

On + ai2 - - ^22 = -2 < 0. (30) 

Thus strategy V is more abundant than C for any mutation rate. 

Surprisingly, the introduction of loners (£), which do not participate in the game 
(Hauert et al., 2002), can dramatically change the balance between C and V. Con- 
sider the following game: 



(31) 





c 


V 


c 






1 





w 


11 


2 







^ 
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Loners are dominated by cooperators and defectors. Elimination of the dominated 
strategy £ leads to a game between C and V, in which V is winning. Thus, standard 
game theoretic arguments predict that strategy V is the most abundant. However, 
these arguments fail for weak selection, where it is not enough to know that a 
strategy dominates another, but also how strong this dominance is. In pairwise in- 
teractions, the advantage of C over jC is significantly larger than that of T> over C as 
can be seen from the matrices: 





(32) 

This advantage of C can overcompensate the disadvantage it has against T>, there- 
fore the abundance of C can be the highest. 

Indeed, the relevant quantities for low mutation rates are 

8 4 

Lc = -, -, and Lc = -4. (33) 

Thus, both C and V have larger abundance than the neutral value 1/3. But since 
Lc > L-p, strategy C has the highest abundance. The introduction of loners causes 
the reversal of abundance between C and V when the mutation rates are small. In 
other words we can say the loners favor cooperators. 

For high mutation rates the relevant quantities are 

Hc = l, Hj, = ^, and = ~. (34) 

Hence, according to (3), both C and T> have an abundance larger than 1/3 for any 
mutation rate. For high mutation rates, however, since He < Hd, strategy V be- 
comes the most abundant. In fact, C is the most abundant for ji < ji* = 2, but it is 
Vfov ii> II*. 



3.2 Reversing the ranking of strategies by mutation 



As a second example, we address the game 





Si 


^2 


^3 
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A 


8 






7 


9 



where A is a free parameter. For A < 7, 5*2 is dominated by S^,. Moreover, 
dominates S2,, and Si and S2 are bistable. Thus, classical game theoretic analysis 
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Fig. 1. Strategy abundance (mean frequency) in the game given by the payoff matrix (35). 
Colored lines show the critical conditions under which one of the three strategies exceeds 
an abundance of 1/3. For small mutation rates, Si is favored over S3, but for large mutation 
rate, S3 is favored over Si. All three strategies have equal abundance at the intersection of 
all boundaries. 

shows that for A < 7, all players should choose Si. It turns out that this state is also 
the only stable fixed point of the replicator equation for A < 7. 

However, the above reasoning does not apply for weak selection. The relevant quan- 
tities for low mutation rates are 

6-A , 2A-9 , , 3-A 

Li = , L2 = — - — , and L3 = , (36) 

and for high mutation rates they are 

4- A 2A-14 , 10 -A 

Hi = ^, H2 = ^-, and H3 = ^-. (37) 

Thus, we expect thresholds where the abundance of a strategy crosses 1/3 at A = 3, 
A = 4.5, and A = 6 for small mutation rates and at A = 4, A = 7, and A = 10 
for high mutation rates. For each mutation rate and each value of A, our conditions 
determine the order of strategies. Fig. 1 shows the change of these thresholds with 
the mutation rate. There are six possibilities for ordering of these three strategies. In 
each of these cases, there can be one or two strategies with an abundance larger than 
1/3. Therefore, there are 12 ways for ordering the strategies relative to 1/3. In this 
concrete example, all of these 12 regions can be obtained by varying the parameter 
A and the mutation rate /x. For example if we fix A = 4.6, just by changing the 
rescaled mutation rate, we obtain six different orderings of the strategies relative to 
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0.1 0.2 0.5 1.0 2.0 5.0 10.0 20.0 

Mutation Rate [i 



Fig. 2. Simulation results for strategy abundances as a function of the reseated mutation 
rate jj, = Nu in the game of payoff matrix (35), at A = 4.6. The population size is = 30 
and the selection strength is (5 = 0.003, which means N6 = 0.09. The solid lines are the 
theoretical curves given by (20), and the dotted line marks the average abundance 1/3. The 
intersections of the lines are located at the critical values given by (3) and (4). The highest 
possible value of the mutation rate at this system size is // = 30, which corresponds to 
mutation probability u = I, where all densities are equal. 

1/3, as one can see in Fig. 1. 

In order to verify our results we performed simulations of the Moran model with the 
payoff matrix (35), at A = 4.6. In figure 2, we compare the simulated frequencies 
of strategies to the theoretical frequencies given by (20). The theory becomes exact 
in the ^ oo, N5 0, and fi = Nu constant limit. As shown in figure 2, already 
at = 30, and 5 = 0.003, which corresponds to N5 = 0.09, we find an excellent 
agreement with the theory. 



3.3 Cooperators, Defectors, and Tit-for-Tat 

As a third example, we discuss the interaction of 'always cooperate' (AUG), 'al- 
ways defect' (AUD), and 'tit-for-tat' (TFT) strategies in the repeated Prisoner's 
Dilemma game (Nowak and Sigmund, 1989; Imhof et al., 2005). Each pair of play- 
ers plays m > 2 rounds. TFT follows its opponent strategy in the previous round, 
but cooperates in the first round. Acting as a cooperator costs c for a player, but one 
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(a) Low mutation rates 



(b) High mutation rates 




Fig. 3. Strategy abundance in the interaction between AIIC, AUD, and TFT in the probabil- 
ity simplex S3. Dark areas ai^e inaccessible to the evolutionary dynamics. Red lines show 
thresholds where a strategy abundance crosses 1/3, the thresholds are given in terms of 
b/c. Blue lines depict thresholds where two strategy abundances are identical, (a) For small 
mutation rates, the abundance of AIIC is never above 1/3 and it is never greater than the 
abundance of TFT. (b) For high mutation rates, the abundance of AIIC is above 1/3 in the 
yellow shaded area, but again it never exceeds the abundance of TFT. 

gets benefit b from playing with a cooperator. Hence, the payoff matrix is given by 



(38) 





AUG 


AIID 


TFT 


AUG 


( {h — c)m 


—cm 


(6 — c)m 


AllD 


bm 





b 


TFT 


\{b — c)m 


— c 


{b — c)m 



For low mutation rates, the relevant quantities are 

2cm 



L 



AUG 



3 



-&(m- 1) + c(3m+ 1) 
-t^AiiD = ^ y^^) 



L 



TFT 



b{m — 1) — c{m + 1) 



The most apparent consequence is that for low mutation rates cooperators never 
exceed the abundance of 1/3. This is not surprising, since AUG is a fairly dull 
strategy: the mean AIID and the cleverer TFT is expected to perform better. As 
we increase the benefit to cost ratio b/c, the order of abundance of these strategies 
change at several particular values. For ^ < only the abundance of AIID is 
larger than 1/3. For i^i^ < ^ < the abundance of both AIID and TFT is 

above 1/3, with AIID still dominating TFT. For ^ > ^^^^ TFT becomes more 
abundant than AIID, for ^ > the abundance of AIID drops below 1/3, and for 
- > ^m+l it is even smaller than the abundance of AUG. 

c m—1 ' 
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For high mutation rates, the relevant quantities are 



b{m — 1) — c(4m — 1) 
iiAiic = ^ 

H^,, = -^Km-l) + c{5m+l) ^^^^ 
9 

b{m — 1) — c(m + 2) 

-"TFT = g ■ 

Surprisingly, now the abundance of AUC can exceed 1/3 for high mutation rates. 
Again, as we increase the benefit to cost ratio b/c, the abundances change order at 
particular b/c values, which values are different for the high and low mutation rate 
limits. For high mutation rates, when ^ < only the abundance of AUD exceeds 
1/3. For ^ < ^ < also the abundance of TFT is larger than 1/3, but does 

' m— 1 c m— 1 ' o / ' 

not exceed the abundance of AUD. For ^^^dil < ^ < ^"^+| , AUD is less abundant 
than TFT. At ^ = 2^m-i) ' abundance of AUD drops below 1/3 and it becomes 
identical to the abundance of AUC at - = Finally, for - > even the 

c m— 1 -' ' c m— 1 ' 

abundance of AUC exceeds 1/3, but it always remains below the abundance of 
TFT. The relations between the strategies and these thresholds are depicted in Fig. 
3. 

The most interesting region is - > where the abundance of AUC exceeds 

1/3 (the yellow region in Fig 3b). This is not possible for low mutation rates. High 
mutation rates and the TFT strategy can facilitate AUC to increase its abundance 
above average. 



4 Outlook 



In this section we discuss possible extensions and limitations of our method. First 
in 4.1 we address the strong selection limit. Then in 4.2 we consider more general 
mutation rates. Finally in 4.3 two alternative dynamics are studied. 



4.1 Strong selection 



Can we say something without the weak selection assumption? As we mentioned 
in Section 2.2, for only two strategies condition (19) is valid for any intensity of 
selection in a wide class of models (Antal et al., 2009). We can also argue that our 
condition (2) is valid for very high mutation probabilities, namely for u 1, for 
arbitrary strength of selection. In this case players pick random strategies most of 
the time, hence the frequencies of all strategies are close to 1/n. This implies that 
the payoff of a A; -player is approximately fk = (1/n) I]i Ofc?. while the total payoff 
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of the whole population is F = {l/n)'^J2i,jO'ij- Strategy k perforais better than 
average when fk > F, which is indeed our general condition for large mutation 
rates (2). Since the system is almost neutral due to high mutation, we hardly need to 
assume anything about the dynamics. Note that u — > 1 implies a stronger mutation 
rate than — > oo, since the latter corresponds to any fixed mutation probability u 
in the — > oo limit. 

The situation is more complex in the low mutation rate limit for arbitrary strength 
of selection. If the mutation rate is sufficiently small we can assume that there are at 
most two strategies present in the system at any given time (Fudenberg and Imhof, 
2006). Then we can use the fixation probabilities, or their large N asymptotic val- 
ues (Antal and Scheuring, 2006; Traulsen et al., 2006), and describe the system 
effectively as a Markov process on n homogeneous strategy states. This descrip- 
tion, however, can lead to very different conditions for arbitrary selection and for 
weak selection. Note also that if two strategies j and k tend to coexist, ajj < a^j 
and ttjk > Qjkfc, the time spent in the mixed strategy state is exponentially large in N 
(Antal and Scheuring, 2006). Hence in this case, the effective Markov process de- 
scription is only valid for extremely small mutation probabilities u <^ e~^^, where 
A is a constant. 



4.2 More general mutation rates 



Throughout this paper we have considered uniform mutations: each strategy mu- 
tates with the same probability m to a random strategy. In this section we extend 
our method to a more general class of mutation rates. For uniform mutation rates 
strategies have equal abundances in the absence of selection, and we have studied 
the effect of selection on this uniform distribution. Conversely, for non-uniform 
mutation rates strategies typically have different abundances already in the absence 
of selection. It can be still of interest to study whether selection increases or de- 
creases these neutral abundances. In principle the perturbation theory presented in 
this paper can be repeated for general mutation probabilities, the discussion how- 
ever becomes unwieldy. 

Here we present an easy generalization to a specific class of mutation rates. Imagine 
that each player mutates with probability u, but instead of uniformly adopting a new 
strategy, it adopts strategy j with probability pj > 0. We can approximate these 
probabilities (up to arbitrary precision) by rational numbers pj — rrij/M, with 
M = J^ifT^i, and all rrij > 1. Then instead of our n-strategy game, we consider an 
M-strategy game, where each original strategy j is represented rrij times. Instead 
of the n X n payoff matrix, it is straightforward to construct the M x M payoff 
matrix, with which all our formulas (1), (2) or (3) automatically apply. 
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4.3 Alternative processes 



Although we have focused on the Moran model in this paper, the results are almost 
identical for the Wright-Fisher (W-F) process and for the Pairwise Comparison pro- 
cess. In the W-F model, each player of a new (non-overlapping) generation chooses 
a parent from the previous generation with probability (abbreviated as w.p.) propor- 
tional to the parent's payoff. The offspring inherits the parent's strategy w.p. 1 — u, 
or adopts a random strategy w.p. u. 

The expected number of offspring of a fc-player in the next generation due to se- 
lection is ujk = Nfk/F, in a given state. In the weak selection limit 5 — > it 
becomes 

= l + 5[(Ax)fc-x^Ax]. (41) 

This is the same as the analog expression (5) for the Moran process, apart from 

the extra N factor. That N factor is due to the definition of time: time is measured 
in single player update steps in the Moran model, while in generations in the W-F 
model. For the neutral correlations, the only difference between the two models 
in the large N limit is that in the W-F model both linages can have mutations in 
each step. Hence all the neutral correlations S2 and S3 are the same as in the Moran 
model of appendix A, provided we use ^ = 2Nu. Consequently, (Aa;^®% becomes 
N times larger than for the Moran process (19), and jj, = 2Nu. 

Taking into account mutations as well, the expected total change of frequency in 
one generation is 

Axf' = Axfil -u) + uQ^-Xk), (42) 

similarly to (10). Hence the average frequency of /c -players in the stationary state 
is 

1 1 — ?/ 

{xk)s^- + {^xf)s, (43) 

n u 

which is identical to (1 1) apart from an extra N factor. Since we also have an extra 
N factor in {Axf^)s for the W-F process, these factors cancel out, and we obtain 
the same stationary density (20) as for the Moran process but with 2Nu instead of 
Nu (similarly to Antal et al. (2008)). This also implies that the condition for greater 
abundance (3) becomes + 2NuHk > 0. 

Conversely, the results are identical for the Moran and the Pairwise Comparison 
process. In this latter model we pick randomly a pair of players, say a type j and a 
type k. The j-player then adopts strategy k w.p. J^{fj — fk), otherwise the fc-player 
adopts strategy j. Here J^{y) = [1 + e''^]"^ is the Fermi function, and the fitnesses 
are defined as f = Ax. The above comparison of the pair of players takes place 
w.p. 1 — u. Instead, w.p. u one of them adopts a random strategy. 

Let us calculate directly the change of the frequency of /c-players due to selection 
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Axf^ in state X. The number of /c-players changes if we pick a A; -player and a 
j k player, which happens w.p. 2xkXj. Then the frequency Xk increases by 
w.p. J^ifj — fk), and decreases by 1/N w.p. J^{fk — fj)- This leads to 

^^f = 1^ E [Hfj - fk) - Hfk - m (44) 

which, in the leading order of small 5, becomes 

^^f - ^ E^.(A - /.) = ^(/^ - E^./.)- (45) 

With the above definition of fitness we arrive at the same expression we obtained 
for the Moran process (6) and (7). Since without selection this model is equivalent 
to the Moran model, all neutral correlations S2 and S3 are also the same. Mutations 
in this model have the same effect as in the Moran model (10). Consequently all 
results we obtained for the Moran model are valid for the Pairwise Comparison 
process as well. 



5 Discussion 



We have studied evolutionary game dynamics in well-mixed populations with n 
strategies. We derive simple linear conditions which hold for the limit of weak 
selection but for any mutation rate. These conditions specify whether a strategy 
is more or less abundant than 1/n in the mutation-selection equilibrium. In the 
absence of selection, the equilibrium abundance of each strategy is 1/n. An abun- 
dance greater than 1/n means that selection favors this strategy. An abundance less 
than 1/n means that selection opposes this strategy. We find that selection favors 
strategy A; if + NuHk > 0, where Lk and Hk are linear functions of the payoff 
values given by eqs (1) and (2). The population size is given by N and the mutation 
probability by u. Furthermore, if Lk + NuHk > Lj + NuHj then the equilibrium 
abundance of strategy k is greater than that of strategy j. In this case, selection 
favors strategy k over j. 

The traditional approach to study deterministic game dynamics in large populations 
is based on the replicator equation (Hofbauer and Sigmund, 1998), which describes 
selection dynamics of the average frequencies of strategies. (Note the formal sim- 
ilarity between (7) and the replicator equation). This method, however, neglects 
fluctuations around the averages. In this paper we have taken into account stochas- 
tic fluctuations, and derived exact results in the limit of weak selection. We find 
the average frequencies of strategies in the stationary state, and conditions for a 
strategy to be more abundant than another strategy. Our conditions are valid for 
arbitrary values of the mutation rates. For small mutation rates these conditions 
describe which strategy has higher fixation probability (Nowak et al., 2004). 
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Throughout the paper we have considered large population size, N, in order to 
simplify the presentation. But in principle all calculations can be performed for any 
given population size N and mutation probability u (see for example Antal et al. 
(2008)). The mutation probability is a parameter between and 1. In a social con- 
text, mutation can also mean 'exploration': people explore the strategy space by 
experimenting with new strategies (Traulsen et al., 2009). A high mutation proba- 
bility seems to be appropriate for social evolutionary dynamics. Our conditions can 
be applied for the initial analysis of any evolutionary game that is specified by an 
n X n payoff matrix. 
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A Probabilities S2 and S3 

This section is valid for any number n > 1 of strategies. We calculate the probabil- 
ities S2 and S3 in the neutral (6 = 0) stationary state. First consider the simpler S2, 
that is the probability that two randomly chosen players have the same strategy. We 
shall use the Moran model and apply coalescent ideas (Kingman, 1982a,b, 2000; 
Wakeley, 2008; Haubold and Wiehe, 2006; Antal et al., 2008). Coalescence means 
that different family lines collide in the past. A key fact behind this idea is that 
there is always a common ancestor of multiple individuals in finite populations. In 
the absence of mutations, any two players have the same strategy in the station- 
ary state, because they both inherit their strategy from their common ancestor. In 
the presence of mutations, two players may have different strategies due to muta- 
tions after the branching of their ancestral lineage. Therefore, tracing the lineage of 
two players backward in time and finding the most recent common ancestor, from 
which two family lines branch, enable us to estimate the similarity of two players 
in strategies. 

Consider two different individuals and let us trace their lineages backward in time. 
In the neutral Moran process, two lineages coalesce in an elementary step of update 
(i.e. two players share the same parent) with probability 2/N'^. Here and thereafter 

we assume that the population size is large, hence we can use a continuous time 
description, where the rescaled time is r = t/{N'^ /2). In the rescaled time, the tra- 
jectories of two players coalesce at rate 1. Following the trajectory of an individual 
back in time, we see that mutations happen at rate ///2 = Nu/2 to each trajectory. 
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The coalescence time T2 is described by the density function 

/2(r2)=e-^^ (A.1) 

Immediately after the coalescence of two players we have two players of the same 

strategy. What is the probability S2{t) that after a fixed time r they have again the 
same strategy? With probability (abbreviated as w.p.) e^'*^ none of them mutated, 
so they still have the same strategy. Otherwise at least one of them mutated, hence 
they have the same strategy w.p. 1/n. The sum of these two probabilities gives 

1 - e"^^ 

S2(t) = e-^^ + ^ . (A.2) 

n 



Now we obtain the stationary probability S2 by integrating this expression with the 
coalescent time density of (A.l) as 

S2 = r s,ir)Mr)dr = . (A.3) 

Jo n[l + iJ,) 



Next we calculate the probability S3 that three randomly chosen players have the 
same strategy. Any two trajectories of three players coalesce at rate 1, hence there 
is a coalescence at rate 3. The coalescence of two out of the three trajectories then 
happens at time T3, described by the density function 

/3(r3) = 3e-3-3. (A.4) 

The remaining two trajectories then coalesce at time T2 earlier, with density func- 
tion (A.l). Before the first coalescence at time T3 backward, the two players have 
the same strategy w.p. S2, and of course they are different w.p. 1 — S2, where S2 is 
given by (A.3). Hence just after this coalescence event we have either three iden- 
tical players w.p. S2, or two identical and one different player otherwise. Now we 
shall see what happens in these two scenarios. 

If we have three identical players then they are also identical after time r w.p. 

stM = ^ [1 + 3(n - l)e-^^ + (n - l)(n - 2)6-'^"^] . (A.5) 



3 , 



To derive this expression note that w.p. e a^"" none of the players have mutated, 
hence they have the same strategy. Then w.p. 3(1 — e~2^)e^^^ one of them has 
mutated, hence they are the same w.p. 1/n. Otherwise at least two of them mutated 
hence they are the same w.p. l/n^. By collecting these terms one obtains (A.5). 

Similarly, if after the first coalescence only two players share the same strategy and 
one has a different strategy, the probability of all three having the same strategy 
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after time r is 



s**{t) = — [l + (n - 3)e-^" - (n - 2)e-i 

n ^ 



(A.6) 



Now we can simply obtain S3 by first integrating over the coalescent time distribu- 
tion (A.4) for the two different initial conditions, and then weighting them with the 
probabilities of the initial conditions, namely 
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